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Introduction 


This report examines the relationship between students’ 
participation in high-impact practices (HIP) and their 
perceived learning gains, with special attention paid to the 
nature of these relationships among Black and Hispanic 
students and among students 25 years or older. In 2018, with 
support from Lumina Foundation, the National Association 
of System Heads (NASH) launched an effort with four 
public college and university systems to demonstrate how 
a coordinated approach could expand and improve quality 
implementation of high-impact practices in an equitable 
manner. In addition to expanding HIP participation for 
traditionally underserved student groups, a key goal of this 
effort was to better understand and measure the gains in 
student learning that can result from participation in 
well-designed high-impact practices and the extent to 
which this learning also occurs equitably. 


Drawing on National Survey of Student Engagement (NSSE) 
data involving more than 28,000 students from 15 institutions 
across three states participating in the Lumina- NASH 
initiative, this report assesses the benefits associated with 
participation in various HIPs in terms of students’ academic 
and practical learning. Overall, analyses point to a substantial 
boost in perceived learning gains for students participating 
in HIPs, with an especially large benefit associated with 
participating in service-learning courses involving a community- 
based project. Participation in HIPs is strongly associated 
with learning gains for Black and Hispanic students as well as 
for students 25 years and older. However, analyses suggest 
that these relationships vary among the different HIPs 
measured by NSSE. 
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Background 


“High-Impact Practices"—a term popularized by the Association of American Colleges and Universities 
(AAC&xU) through its Liberal Education and America’s Promise (LEAP) initiative and researched over 
time by many researchers using data from NSSE and elsewhere—are a set of educational practices that 
result in higher student engagement.' The most well-known HIPs are those promoted by AAC&U that 
include first-year seminars, common intellectual experiences, learning communities, writing-intensive 
courses, collaborative assignments and projects, undergraduate research, diversity and global learning, 
e-portfolios, service learning or community-based learning, internships, and culminating senior 
experiences or capstone projects.” 


For over a decade, researchers have developed a robust evidence base on the benefits of high-impact 
practices for students in terms of their academic engagement, attainment, and learning outcomes,’ 
leading many colleges and universities across the country to promote, scale, and even require student 
participation in HIPs. Importantly, some studies have suggested that HIPs may be particularly beneficial 
for minoritized students, first-generation students, and other student groups that tend to be underserved 
within higher education.* However, other studies have called into question the assumption that HIPs 
benefit all students equally. Some have even raised concerns that HIPs may result in negative experiences 
for minoritized students by creating more opportunities for exposure to racist interactions.* 


This study builds on previous research by examining the relationship between participation in HIPs 
and student learning gains, paying attention to variation by HIP type as well as in the nature of the 
learning gains connected to HIP participation. We explore these relationships for all students as well 
as for minoritized students and for students 25 and older. 


Data and Methods 


Data used in the analysis derive from NSSE surveys administered between 2014 and 2020 at 15 
four-year institutions in Georgia, Wisconsin, and Montana, and include approximately 28,500 students.° 


Table 1: Sample Characteristics, n=28,516 


Female 67% Average age 23 
Male 33% 25+ 19% 


SEs se 
Whi 


Black 14% Freshman 46% 

Hispanic 7% Senior 54% 

American Indian 0.4% 

at % _-—-—*Fitst-Generation & Transfer status 

Other 6.3% First-Gen student AT% 
Transfer student 30% 
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Table 1 provides descriptive information for the analytic sample. Two-thirds of students in the sample 
are female. Approximately one-half of those in the sample are classified as freshmen and the other half 
as seniors. Roughly two-thirds of students are white, 14 percent are Black, 7 percent are Hispanic, and 
less than 0.5 percent are American Indian or Alaska Native. The average age of students in the sample 
is 23 years old, with approximately one in five students 25 years or older. Finally, nearly half of 
students are first-generation, and 30 percent are transfer students. 


Measures of HIP Participation 


Measures of HIP participation are based on NSSE indicators of participation in six HIPs. NSSE asks 
student respondents about their participation in the following high-impact practices: learning commu- 
nities, service-learning courses involving a community-based project, research with faculty, internships 
or field experiences, study abroad, and culminating senior experiences or capstone projects.’ 


The HIPs that students engage in vary in nature and approach and can therefore be expected to 
correlate with different types of learning. For example, research with faculty may help students hone 
their analytic reasoning skills, whereas study abroad may help students develop deeper awareness 

of other cultures and backgrounds. Among the six HIPs that survey respondents are asked about in 
NSSE, three relate to experiences that explicitly extend to communities beyond the campus: namely, 
study abroad, which extends to other cultures and geographies; internships, which extend into the 
world of work; and service learning which by definition involves a community-based project. The other 
three H]Ps—learning communities, research with faculty, and culminating senior experiences—may 
also include engagement beyond campus; however, in general, they emphasize on-campus engagement 
with other students and faculty, and they focus on the integration of campus-based learning. 


In order to explore the extent to which these two types of HIPs may differentially affect student 
learning, two “meta-HIP" measures were created for this analysis: campus-based HIPs represent 
participation in learning communities, research with faculty, or a culminating experience, whereas 
community-based HIPs include participation in an internship, a service-learning course involving 
a community-based project, or study abroad (see Figure 1).* 


Figure 2 presents students’ self-reported participation rates in each HIP category as well as in each 
individual HIP for the sample of 15 institutions. Overall, more than two-thirds of students in these 
institutions report participating in at least one community-based HIP (68 percent), which is driven by 
high participation rates in service learning (57 percent), moderate participation in internships (31 percent), 


Figure 1: Authors’ Categorization of High-lmpact Practices Included in NSSE 


Campus-Based HIPs Learning Communities 


Research with Faculty 
Culminating Senior Experience or Capstone 


Service-Learning Courses Involving a Community-Based Project 
Internship or Field Experience 
Study Abroad 
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Figure 2: HIP Participation Rates 
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and lower participation in study abroad (9 percent). Campus-based HIPs are experienced by more 
than 40 percent of students, with highest participation in culminating senior experience or capstone (25 
percent), followed by learning community (20 percent) and research with faculty (14 percent). 


Some differences in participation patterns emerge when examining HIP participation by students’ race, 
ethnicity, and age. Among community-based HIPs, Black and Hispanic students have slightly higher 
participation rates in service learning compared to the full sample, whereas these students are less likely to 
participate in internships. Students 25 or older exhibit the opposite pattern: They participate in service 
learning at slightly lower rates compared to the full sample, and they participate in internships at higher rates. 


Although participation in study abroad is relatively low among all student groups, Black students and 
students 25 and older participate in this HIP at lower rates compared to the full sample. Among 
campus-based HIPs, students 25 and older are more likely to participate in a culminating senior 
experience or capstone; in contrast, participation rates in culminating senior experiences by both 
Black and Hispanic students are lower compared with the overall sample. Black and Hispanic students 
also participate in research with faculty at slightly lower rates than the full student sample. Finally, 
Black students in the sample participate in learning communities at higher rates, and students 25 and 
older do so at lower rates compared to the full sample; Hispanic students’ participation rates in 
learning communities are comparable to that of the full sample. 


Measures of Gains in Student Learning 


In addition to questions on HIP participation, NSSE also asks respondents to assess the extent to which their 
college experience contributed to their knowledge, skills, and personal development in a variety of domains, 
including those that are more academic in nature as well as those that are more practical in nature or 

related to personal or social development. In total, the survey asks respondents to answer 10 questions 
related to perceived gains.’ These questions were used to create three primary outcome measures for the 
analysis: overall learning gains, academic learning gains, and practical learning gains (see Figure 3). 
Each of the three scale measures averages students’ reported scores across the relevant learning gains items.'° 
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Practical Domains 


NSSE asks student respondents to assess the extent to which their experience at the institution contributed to their knowledge, skills, and 
personal development in 10 areas. For each area, students rate their perceived gains on a four-point scale (very little, some, quite a bit, very much). 


Writing clearly and effectively 
Speaking clearly and effectively 

Thinking critically and analytically 

Analyzing numerical and statistical information 


Acquiring job- or work-related knowledge and skills 
Working effectively with others 

Developing/clarifying a personal code of values and ethics 
Understanding people of other backgrounds 
Solving complex real-world problems 
Being an informed and active citizen 
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Estimation Strategy 


To estimate the relationship between HIP participation and perceived learning gains, a series of ordinary 
least squares (OLS) regressions were conducted in which each of the three perceived learning gains scales 
(overall, academic, practical) were regressed on indicators of HIP participation. This included both models 
in which all individual HIPs are included as explanatory variables, and models in which the two 
“meta-HIPs" (campus-based HIPs and community-based HIPs) are included as explanatory variables. 
All regressions control for factors that are potentially associated with both participation in HIPs and with 
perceived learning gains, which could bias the association between the two, including students’ race and 
ethnicity, gender, age, class (freshmen vs. senior), first-generation student status, and transfer status.'! 


For all analyses, perceived learning gains scales are mean-centered and standardized (i.e., subtracted 
the mean value and divided by the standard deviation) such that each scale has a mean of zero and 
standard deviation of one. The resulting coefficients from each of the models can therefore be inter- 
preted as standard deviation differences or effect sizes.” 


Results 


All Students 


The estimated associations between participation in community-based or campus-based HIPs and 
perceived learning gains are summarized in Figure 4a, and the estimated associations between partici- 
pation in each of the six individual HIPs and perceived learning gains are summarized in Figure 4b. 
The bars in the figures represent the estimated relationship between a given HIP (or HIP type) and 
each of the three perceived learning gains scales (overall learning gains scale, academic gains scale, 
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n=28,516. All coefficients are statistically significant at p<.10. 
All results are from OLS regression models that control for student-level characteristics as well as institution-year fixed effects. 
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practical gains scale), which derive from regression models run on each learning gains scale separately. 
The associations are in standard deviation units, as described above. 


Overall results suggest that students’ participation in both campus-based and community- 
based HIPs is strongly correlated with perceived learning gains. 


As shown in Figure 4a, student participation in both community-based HIPs and campus-based HIPs 
is positively and significantly associated with all three learning gains scales: overall learning, academic 
learning, and practical learning. Across the three scales, participation in HIPs is correlated with an 
increase in learning ranging from 19 percent to 36 percent of a standard deviation. 


Community-based HIPs are more strongly connected with students’ practical learning gains, 
whereas two of three campus-based HIPs are more strongly associated with academic gains. 


As shown in Figure 4a, students participating in community-based HIPs report larger increases in their 
practical learning (medium-color bars) compared to their academic learning (darkest bars). This pattern 
holds for all three community-based HIPs of service learning, internships, and study abroad, as 
illustrated in Figure 4b. By comparison, participation in campus-based HIPs is correlated with roughly 
equivalent increases in both academic and practical learning gains (Figure 4a), though this pattern 
varies among the three HIPs of this type. Specifically, as shown in Figure 4b, the campus-based HIPs 
of research with faculty and culminating senior experience or capstone are more strongly associated 
with academic learning gains, whereas participation in learning communities is associated with a larger 
boost to practical learning gains, a finding more consistent with community-based HIPs. 


Figure 4b: “Boost” to Learning Gains from Participation in Each Individual HIP 
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n=28,516. Coefficients in bold are statistically significant at p<.10. 
All results are from OLS regression models that control for student-level characteristics as well as institution-year fixed effects. 
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Table 2: “Boost” to Learning Gains from Participation in Individual HIPs 


for Black Students, Hispanic Students, and Students 25 and Older 


BLACK STUDENTS HISPANIC STUDENTS STUDENTS 25+ 
Gains Gains Gains 
Overall Overall Overall 
Research 0.14 0.15 0.08 
Culminating 
Experience/ 0.13 0.25 0.17 
Capstone 
Learning 
Conant 0.16 0.13 0.19 
Seniee: 0.30 0.37 0.45 
Learning ' : : 
Internship 0.20 -0.00 0.11 
Study Abroad -0.00 0.13 0.02 


n=4,033 (Black Students), n=1,892 (Hispanic Students), n=5,485 (Adults 25+), Coefficients in bold are statistically significant at p<.10. All results are from 
OLS regression models that control for student-level characteristics and institution-year fixed effects. 


Service learning and research with faculty are associated with the most notable overall 
“boosts” in learning gains. 


As shown in Figure 4b, with the exception of study abroad, participation in each of the six HIPs is 
associated with positive and statistically significant increases in all three learning gains scales (overall 
gains, academic gains, practical gains). Among the six HIPs measured by NSSE, service learning with 
a community-based project is the HIP that confers the largest boost to student learning in terms of 
both academic and practical learning gains. This is notable given that more students participate in 
service learning than in any other HIP, both in these data (see Figure 2) and in national NSSE data. 
In addition, students engaging in research with faculty report notable gains in their academic learning. 


Black Students, Hispanic Students, and Students 25 and Older 


A key goal of the Lumina-NASH HIPs initiative was to better understand and measure the extent to 
which student learning from HIPs occurs equitably. In addition to exploring relationships between 
HIP participation and learning gains for the full student sample, separate regressions were conducted 
for Black students, Hispanic students, and students 25 years and older.'? For these students, Figure 5 
displays the “boost" to learning gains associated with participation in community-based HIPs or 
campus-based HIPs as categories, and Table 2 summarizes the relationship between learning gains 
and participation in each individual HIP for these student groups. 
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Black students, Hispanic students, and students 25 and older who participate in HIPs 
experience overall positive, significant gains in their self-reported learning. 


Similar to results for all students, Figure 5 points to statistically significant gains in learning for Black 
students, Hispanic students, and students 25 and older who participate in both community-based and 
campus-based HIPs. These results support earlier studies pointing to generally positive benefits of 
HIPs for minoritized and underserved student groups in higher education. 


Learning gains for Black and Hispanic students who participated in community-based HIPs 
are large and driven by service learning. Black students also experience notable benefits 
from participation in internships, and Hispanic students from study abroad. 


Black and Hispanic students engaging in community-based HIPs report notable increases in learning 
gains, especially practical learning gains. However, there is considerable variation in patterns for Black 
and Hispanic students among the three community-based HIPs. Specifically, Black students in this 
study have an especially large “boost” to learning gains from participation in internships but do not 
have statistically detectable learning gains associated with study abroad experiences. In contrast, 
Hispanic students in this sample report notable gains from study abroad yet do not appear to have 
learning gains from internships. Both Black and Hispanic students reap large learning gains from 
participation in service learning. 


Students 25 and older report large learning gains associated with participation in 
community-based HIPs, particularly in terms of practical learning. 


Students 25 and older who engage in community-based HIPs experience significant gains in both 
academic and practical learning. This is driven by a notably larger association among these adult 
students between service-learning participation and self-reported gains. Older students also benefit 
from internships though do not report learning gains from study abroad. 


Black students, Hispanic students, and students 25 and older who participated in 
campus-based HIPs also reported increased overall, academic, and practical learning 
gains; however, academic gains associated with research with faculty varied among these 
student groups. 


As shown in Figure 5, Hispanic students report strong overall, practical, and academic learning gains 
from participation in campus-based HIPs. Focusing on academic gains, Hispanic students experience 
an especially large boost from both research with faculty and culminating senior experiences. Black 
students and older students also benefit from campus-based HIPs, reporting positive and significant 
gains from all three HIPs, with the largest overall learning gains from learning communities. However, 
compared to Hispanic students (and compared to all students), Black students and students 25 and 
older in our sample experience a relatively lower boost to academic learning gains from participation 
in research with faculty. 
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Figure 5: “Boost” to Learning Gains from Participation in Community-Based 


and Campus-Based HIPs for Black Students, Hispanic Students, and 
Students 25 and Older 
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n=28,516 (All Students), n=4,033 (Black Students), n=1,892 (Hispanic Students), n=5,485 (Adults 25+), All coefficients are statistically significant at p<.10. 
All results are from OLS regression models that control for student-level characteristics and institution-year fixed effects. 
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Conclusion: 


This analysis reinforces prior research underscoring the educational benefits of high-impact practices. 
It also extends the evidence base by illuminating variation in these relationships both in terms of HIP 
type as well as in the nature of the learning gains associated with HIP participation. 


Results show that participation in community-based HIPs—defined as HIPs that explicitly extend to 
communities beyond the campus—correlates strongly with students’ overall, academic, and practical 
learning gains, and that practical learning gains are especially large for students experiencing these types 
of HIPs. Similarly, campus-based HIPs as a category are connected with positive overall, academic, and 
practical learning gains, though not as pronounced as the reported learning gains from community-based 
HIPs. Reported learning gains associated with both community-based and campus-based HIPs are 
large and significant for all students, as well as for Black students, Hispanic students, and students 25 
years and older. These results reinforce the importance of institutional and system efforts to continue 
equitable expansion of both HIP types. 


Further examination of these relationships for individual HIPs points to larger returns to some HIPs 
and lower returns to others, and to variation in these relationships for Black, Hispanic, and older 
students. Among community-based HIPs, service learning—which is experienced by a majority of 
students in this sample and across four-year colleges nation- 
wide—is connected with large and significant learning gains 


across all student groups—with particularly large benefits for These results reinforce 
students 25 and older. the importance of institu- 

tional and system efforts 
In contrast, study abroad is the HIP least consistently associated to continue equitable 
with learning gains; however, Hispanic students in our sample expansion of HIPs. 


appear to derive substantively meaningful benefits from these 

experiences. Black students report notable gains from internships compared to all students and to 
other student groups in this analysis. Among campus-based HIPs, participation in learning communities, 
research with faculty, and culminating senior experiences yield overall learning gains for all students. 
However, evidence from this sample suggests that the boost to academic learning gains from research 
with faculty may not be experienced equitably by Black students and students 25 and older. 


Although these findings related to HIPs are specific to the set of 15 institutions included in the analysis, 
they carry broader practical implications for the equitable, quality-conscious implementation of 
high-impact practices. For example, our finding that Black students have relatively weaker correlations 
between participation in research with faculty and gains in their academic learning supports prior 
research suggesting that key aspects of this HIP need to be strengthened for Black students. Specifically, 
prior research shows that Black students are less likely than other students to associate their experiences 
in faculty-assisted research with effective teaching practices and with a supportive environment.'* More- 
over, our overall finding of a relatively weak association between participation in study abroad and 
students’ perceived learning gains points to the importance of ensuring that study abroad programs 
incorporate standards for good practice and are designed to offer students high-quality international 
experiences.'* Study abroad professionals can develop taxonomies, such as those developed by 
IUPUI's Study Abroad office, that aim to identify and measure key attributes deemed essential for 
maximizing the overseas experience to enhance students’ intercultural learning. 
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